Sensors that acquire 3D data are useful for many applications. For example, a system for automated ‘bin-picking’ in a factory can acquire 3D data of a bin containing multiple instances of the same object, and compare the 3D data with a known 3D model of the object, in order to determine the poses of objects in the bin. Then, a robot arm can be directed to retrieve a selected one of the objects. The pose of an object is its 3D location and 3D orientation at the location. One set of vision-based techniques for sensing 3D data assumes that the objects have non-specular surfaces, such as matte surfaces.
Another type of sensor determines the silhouette of the object, and compares the silhouette with the known 3D model of the object, in order to determine pose. One set of techniques for determining the silhouette assumes that, the objects cast shadows when illuminated.
Non-Specular Surfaces
Vision-based techniques for sensing 3D data for non-specular surfaces include structured light, time-of-flight laser scanners, stereo cameras, moving cameras, photometric stereo, shape-from -shading, and depth-from-(de)focus.
All of these techniques assume either that incident: light on the surface is reflected diffusely, and hence, reflected light is visible at: any sensor with a line-of-sight to the surface or they assume that visible features are actually physical features on the object surface with a measurable 3D physical location, and are not reflected features. These techniques degrade as the surface becomes less diffuse and more specular, because the above assumptions are no longer true.
Specular Surfaces
Vision-based techniques for sensing the 3D pose and shape of specular surfaces assume that there are features in a surrounding scene that are reflected by the specular surface. The features may be sparse, such as specular highlights arising: from point light sources in the scene. If the features are sparse, then the sensed 3D shape of the surface is also sparse. This is undesirable for many applications. For example, it is difficult to determine a reliable pose of an object when the sensed features are sparse. The problem can be ameliorated by moving the camera or the identifying features relative to the surface, but this increases the complexity of the system and is time-consuming.
Semi-Specular Surfaces
There are few vision-based sensors known in the art for objects with semi-specular surfaces, such as bin shed metal where the surface reflects some of the incident light in a specular way, and some of the light in a diffuse way. This is because techniques that sense 3D data by using diffuse reflection, receive less signal from a semi-specular surface, so they are less reliable. The techniques that determine the object silhouette using cast-shadows are also less reliable because the shadow is less pronounced when it is cast on semi-specular background, as occurs with a bin of semi-specular objects for example. Techniques that work on specular objects are inapplicable because sharp reflected features are not visible.
Thus, there is a need for a method and system for determining poses of semi-specular objects that performs well on varied surface shapes such as planar and curved semi-specular surfaces.